For Orthopedic Care, Hospital Ranking Systems Can Be Misleading

Patients traditionally rely on hospital comparison charts and rating systems for guidance on quality care, but hospital ratings specific to orthopedic surgery can be misleading due to a lack of stated methodology, according to research published in the Journal of the American Academy of Orthopaedic Surgeons (JAAOS). The new study investigated five hospital rating systems to assess consistency and agreement among the hospitals deemed “high-performing” and “low-performing” for orthopedic surgery. It revealed that the systems generally rate hospitals based on varied data sources and different methodologies that lack robust formal validation. Thus, they are subject to misrepresentation that can lead to alienated, frustrated and confused patients.
 
“Hospital ranking systems are valuable consumer tools for information on the safety and quality of our nation’s hospitals,” said lead author and vice chair of the Department of Orthopedics at Northwestern Medicine, David W. Manning, M.D., FAAOS. “However, our research found there to be very little consistency between rating systems specific to the orthopaedic specialty. In fact, it was more likely that any given hospital would be rated as both ‘high-performing’ and ‘low-performing’ for quality across multiple rating systems than it was for any one hospital to be ranked as ‘high-performing’ across the board.”
 
Although hospital rating systems are not new, they have become increasingly available to the public and replied upon by patients. Most rating systems base the results on four basic groups of data: structure, process, outcomes and reputational scores. Reasons for the varied results across rating systems include weighting each of these groups of data differently, opposing data sources and various methodologies for processing the data within each group. Because there is no universally accepted method for measuring or reporting excellence in orthopedic surgery, each rating system is designed to support the missions of the companies that create them. Each has a unique method for data procurement and processing which results in inconsistencies across hospital rating systems.
 
According to Dr. Manning, with such variability between hospital rating systems, the responsibilities fall to individual institutions and physicians to report their own outcomes with transparency.
 
“Inconsistency among these rating systems makes it more important for us to own the conversation with our patients by providing patient commentary regarding our own performance and our own internal quality metrics for any given procedure, rather than having the rating systems do that for us,” Dr. Manning added.
 
For patients seeking specific orthopedic treatment or assistance, Dr. Manning suggests caution when using ranking systems as the be-all-end-all, and shares the following tips to assessing current and future orthopaedic care: 

• Don’t rely solely on rating systems 
• Look for hospital and physician self-reported quality metrics and compare to local competitors 
• Ask friends and family for personal experiences and recommendations to validate your findings 
• Refer to your general physician for advice 
• Trust your instincts

 
The review article notes that further research is needed regarding rating systems that assess not only orthopedic surgery programs, but hospitals and providers, as well. There are many aspects of healthcare that are not addressed by these rating systems, and patients should use ratings as a jumping-off point in evaluating orthopedic specialty care.
 
“In the future, these rating systems can only get better,” Dr. Manning said. “I believe there is a real desire among American healthcare seekers to have quality information about where and from whom they should receive their care. Eventually, there will be an acceptable definition of what constitutes excellent healthcare quality and it will be available to patients to freely review.”